1. Field of the Invention
The present invention relates to a method of compensating for the overall brightness variation between image frames required for image processing such as in video editing and coding for efficient transmission and storage of digital video images.
2. Background Art
As a conventional coding technique for efficiently transmitting and storing video image data, a technique known as motion compensation interframe prediction coding has been widely used. With this technique, the luminance values (or color differences) of the image frames being processed are not directly transmitted and stored; instead, the technique measures how much objects inside an image have moved between a video image frame which has already been coded (referred to as a reference image frame) and the image frame being processed, and luminance difference values at positions which are shifted by the distance of movement are transmitted and stored.
Movement compensation interframe prediction coding enables transmission and storage of reduced quantities of data because the difference values are generally less than the luminance values themselves.
In this case, when measuring movement, it is usually assumed that the image frame being processed and the reference image frame have the same lighting conditions. That is, movement is detected under the assumption that corresponding objects in both frames have the same luminance values. However, the luminance values for corresponding objects can differ considerably between frames in cases such as when there is flickering, when a strobe light has flashed, or when the entire image is made gradually brighter (hereinafter referred to as “fade-in”) or darker (hereinafter referred to as “fade-out”) by adjusting the diaphragm of the camera or by using video editing equipment. Consequently, the following problems are associated with motion compensation interframe prediction coding:                (1) The detection of movement is not properly performed.        (2) The coding efficiency is not sufficiently improved.        
As a conventional method principally for resolving the above-mentioned problem (1), there is a technique of measuring luminance changes between image frames, and performing movement measurements while correcting for the measured luminance changes.
Luminance changes can be largely separated into types wherein the degree of change changes according to the spatial position in the image frame, such as those which are caused by noise, and types wherein the degree of change is the same over the entire image frame, such as those which are caused by fade-ins and fade-outs.
Since it is important to measure the luminance changes as accurately as possible when attempting to resolve problem (1), the measurements are usually made in small areas within the frame in order to handle both types of luminance changes. However, although this resolves problem (1), the size of the interframe luminance difference values can still result in problem (2).
Thus, there is also a technique of reducing the interframe luminance difference values by transmitting and storing the luminance changes of the small areas as coded data, and correcting for these luminance changes at the decoder side. However, since this technique involves sending coded data expressing the luminance change for each small area, the quantities of data are ultimately increased to such a degree that the overall data quantity is not much different. In other words, the coding efficiency is not much improved.
On the other hand, there is a technique of measuring a single luminance change for the image frame as a whole, and transmitting and storing that luminance change as coded data for the purposes of improving the coding efficiency when the same luminance change has occurred over the entire image frame. With this technique, the quantity of coded data is extremely small because it is sufficient to send only a single piece of coded data expressing the luminance change for each image frame. Additionally, this technique contributes largely to the improvement of coding efficiency in cases of fade-ins and fade-outs because the interframe luminance change values can be made somewhat smaller simply by correcting for the luminance change.
As a specific parameter for expressing the overall luminance change, this technique uses only a parameter expressing the gain difference. That is, taking a parameter DB representing the gain difference, it is assumed that the luminance value x of each pixel will change to x′ as shown in Equation 1:x′=x+DB  (1) In this case, the luminance change _x is always constant:                                                                         Δ                ⁢                                                                   ⁢                x                            =                            ⁢                                                x                  ′                                -                x                                                                                        =                            ⁢                              D                B                                                                        (        2        )            
FIG. 3 shows the changes in the luminance values in an actual case wherein a still image containing various brightness values was taken by a camera and the diaphragm of the camera was gradually closed so as to darken the overall image, i.e. a fade-out. When considering the changes in the luminance values from frame 1 to frame 11, or frame 11 to frame 21, the luminosity changes are approximately constant regardless of the luminance values themselves. Therefore, in this case, the assumption of Equation (1) holds. That is, the constant amount of change corresponds to DB in Equation (1).
On the other hand, FIG. 4 shows the changes in the luminance values in a case wherein the same still image was taken without changing the diaphragm of the camera, then a fade-out was caused using digital editing equipment. When considering the changes in the luminance values from frame 41 to frame 51, or frame 51 to frame 61, the luminance changes change depending on the luminance values. Therefore, the assumption of Equation (1) does not hold in this case, and the precision is insufficient to express overall luminance changes using only the parameter DB.
When the images are divided into small areas of 8×8 pixels, the variations in the luminances of these small areas are generally not very large. For example, when the luminance value in the image frame overall changes from 0-255, the variations in the small areas are almost always in the range of 20-40. In this range, the luminance change can be considered to be approximately constant without regard to the luminance values themselves. That is, the assumption of Equation (1) is still appropriate if the luminance change is found only in small areas.
However, the range of luminance values broadens when the image frame is taken as a whole. That is, there is a considerable possibility that there will be luminance changes covering the entire range of luminance values 0-255. Therefore, the approximation of Equation (1) is insufficient for determining one type of luminance change over the entire image frame.